Skip to content

fix: align graph runtime API with runtime namespace#13

Merged
voltjia merged 6 commits into
feat/graph-runtime-apifrom
fix/graph-runtime-api-runtime-namespace
Jul 3, 2026
Merged

fix: align graph runtime API with runtime namespace#13
voltjia merged 6 commits into
feat/graph-runtime-apifrom
fix/graph-runtime-api-runtime-namespace

Conversation

@voltjia

@voltjia voltjia commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Move generated C++ runtime wrappers under infini::rt::runtime so that API names matching CUDA Runtime API stay in that nested namespace.
  • Keep the outer infini::rt namespace focused on runtime-device selection APIs.
  • Adapt graph C++ tests and C API implementation to the runtime namespace; C API graph handles remain opaque wrappers around backend/raw handles.
  • Generate graph runtime declarations only when an enabled backend implements graph APIs, so CPU-only builds do not expose unsupported graph symbols.
  • Refresh docs and install-consumer tests for the new public API layout.

Motivation

This helper PR lets feat/graph-runtime-api absorb the current master runtime API layout before the graph-runtime PR is rebased. It preserves the project decision that CUDA-aligned Runtime API names live under infini::rt::runtime, while InfiniRT-specific selection APIs live directly under infini::rt.

No issue is closed directly by this PR.

Type of Change

  • feat - new feature / new operator / new platform
  • fix - bug fix
  • perf - performance improvement (no behavioral change)
  • refactor - code restructuring without behavior change
  • test - adding or fixing tests only
  • docs - documentation only
  • build / ci - build system or CI configuration
  • chore - tooling, formatting, or other non-code changes
  • Breaking change (requires a ! in the Conventional Commits prefix or a BREAKING CHANGE: footer)

Platforms Affected

  • CPU (WITH_CPU)
  • NVIDIA (WITH_NVIDIA)
  • Iluvatar (WITH_ILUVATAR)
  • MetaX (WITH_METAX)
  • Cambricon (WITH_CAMBRICON)
  • Moore (WITH_MOORE)
  • Ascend (WITH_ASCEND)
  • PyTorch C++ bindings (WITH_TORCH)
  • Build system / CMake / CI
  • Python bindings / user-facing API

Smoke Test Result

# ssh nvidia, image accelerator-dev/nvidia:latest
cmake -B build -DWITH_CPU=ON -DWITH_NVIDIA=ON \
  -DINFINI_RT_BUILD_TESTING=ON \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build -j$(nproc)
ctest --test-dir build --output-on-failure

100% tests passed, 0 tests failed out of 9
# CPU-only generated API check, same image
cmake -B build-cpu -DWITH_CPU=ON \
  -DINFINI_RT_BUILD_TESTING=ON \
  -DCMAKE_BUILD_TYPE=Release
cmake --build build-cpu -j8
ctest --test-dir build-cpu --output-on-failure

100% tests passed, 0 tests failed out of 6
clang-format 21 dry-run: passed
ruff format --check scripts/generate_public_headers.py: passed
# Integration check with simulated #13 merged into #9 and rebased onto master,
# InfiniCore#1351, InfiniLM commit 36e2207, model Qwen3-0.6B.
python3 examples/test_infer.py --device nvidia --enable-graph \
  --model=/data-aisoft/mechdancer/models/Qwen3-0.6B/

Standalone InfiniRT graph bridge loaded: /workspace/verify/infini-rt-prefix/lib/libinfinirt.so
Using standalone InfiniRT graph bridge for graph capture and replay.
[codex-verify] InfiniRT C API infiniRtStreamBeginCapture: 1
[codex-verify] InfiniRT C API infiniRtStreamEndCapture: 1
[codex-verify] InfiniRT C API infiniRtGraphInstantiate: 1
[codex-verify] InfiniRT C API infiniRtGraphLaunch: 511
Inference completed, total_time: 1448.2 ms

Test Results on Supported Platforms

Platform Affected Build / Smoke Result Full Result / Notes
NVIDIA Yes smoke passed Full graph integration passed with InfiniCore#1351 and InfiniLM as above.
Iluvatar Yes not rerun in this helper PR API/generation touched; backend validation to run on the rebased #9 branch.
MetaX Yes not rerun in this helper PR API/generation touched; backend validation to run on the rebased #9 branch.
Cambricon Yes not rerun in this helper PR API/generation touched; backend validation to run on the rebased #9 branch.
Moore Yes not rerun in this helper PR API/generation touched; backend validation to run on the rebased #9 branch.
Ascend Yes not rerun in this helper PR API/generation touched; backend validation to run on the rebased #9 branch.
Full `pytest` output (optional)
N/A - no pytest suite in this PR.

Benchmark / Performance Impact

N/A - this PR changes API placement, wrapper generation, tests, and docs; it is not performance-sensitive.

Notes for Reviewers

  • This is a helper PR targeting feat/graph-runtime-api, not master directly.
  • It intentionally keeps CUDA-aligned names under infini::rt::runtime; outer infini::rt remains available for InfiniRT-specific device-type selection APIs.
  • Current InfiniLM main has newer InfiniCore RoPE/mrope API expectations than InfiniCore#1351, so the graph integration smoke used InfiniLM commit 36e2207ae6eaa4296b4564895648d50aaeef5453, which is compatible with #1351.

voltjia and others added 2 commits July 3, 2026 09:18
* Align runtime API with generated wrappers

* Add default runtime dispatch specialization

* Refactor runtime dispatch namespace

* Use Abseil status for runtime device API

* Revert "Use Abseil status for runtime device API"

This reverts commit a26ddff.

* Address runtime dispatch review feedback

* Keep runtime API list in generator

* Add TensorView constructor guard test

* Align runtime memcpy kind constants with CUDA API

* Use CUDA-style runtime memcpy constants

* Use CUDA-style runtime memcpy constants

* Move TensorView tests back into core test

* Remove standalone TensorView test target

* Remove standalone TensorView test file

* Use fully qualified runtime API names in README

* style: format runtime dispatch test
@voltjia voltjia force-pushed the fix/graph-runtime-api-runtime-namespace branch from 87fb536 to 943be4b Compare July 3, 2026 02:06
spike-zhu and others added 4 commits July 3, 2026 11:15
Co-authored-by: Jiacheng Huang <huangjiacheng0709@outlook.com>
# Conflicts:
#	scripts/generate_public_headers.py
#	src/native/cuda/iluvatar/runtime_.h
#	src/native/cuda/metax/runtime_.h
#	src/native/cuda/moore/runtime_.h
#	src/native/cuda/nvidia/runtime_.h
#	tests/CMakeLists.txt
@voltjia voltjia merged commit 7036180 into feat/graph-runtime-api Jul 3, 2026
4 checks passed
@voltjia voltjia deleted the fix/graph-runtime-api-runtime-namespace branch July 3, 2026 08:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants